Papers with memory-augmented LMs
Training Language Models with Memory Augmentation (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for training memory-augmented language models only introduce mem-ories at testing time or represent them using a separately trained encoder. |
| Approach: | They propose a training approach that directly takes in-batch examples as accessible memory and new methods for memory construction and data batching that are used for adapting to different sets of memories at testing time. |
| Outcome: | The proposed approach reduces perplexity from 18.70 to 15.37 on multiple language modeling and machine translation benchmarks. |